p-value

2022-03-09 ยท 2 min read

    see: https://www.wikiwand.com/en/P-value

    \[ \renewcommand{\Pr}[1]{\text{Pr}\left[ #1 \right]} \def\eps{\varepsilon} \def\E{\mathbb{E}} \def\pv{p\text{-value}} \]

    Overview #

    • The p-value is a r.v.
    • For a given observation, it's the probability of obtaining observing results at least as "extreme" as the currently observed result, given the null hypothesis is correct.
    • Imagine $f(x)$ is some criterion r.v. which returns the probability of finding more extreme results than some observation $x$. In this case, the null hypothesis is also a distribution which we're assuming $x$ is sampled from.
    \[ \pv(x) = p \in [0, 1] \;:\; \Pr{f(x) \ge p \;|\; x \sim H_0} \]
    • Used in null-hypothesis significance testing.
    • Different from the transposed conditional probability
    \[ \times \;\; \Pr{x \sim H_0 \;|\; f(x) \ge \pv} \] \[ \begin{align*} \Pr{x \sim H_0 \;|\; f(x) \ge \pv} &= \Pr{f(x) \ge \pv \;|\; x \sim H_0} \cdot \frac{\Pr{f(x) \ge \pv}}{\Pr{x \sim H_0}} \\ \end{align*} \]

    Traps #

    1. p-value is conditional probability, not probability the null-hypothesis is true nor probability alternative-hypothesis is false.
    2. p-value is only statement about the relation of the observed data to the null-hypothesis.
    3. p-value significance of 0.05 is only convention.
    4. p-value doesn't indicate size or importance of an effect.